Method for unified collection of content analytic data

ABSTRACT

A method for storing a user evaluation of distributed content comprising an Evaluation Program (EP) that creates a user session when a user accesses distributed content. The user evaluation is saved with other information such as the time and date of the evaluation, the user&#39;s personal information, and the navigation path the user used to access the distributed content page. The EP records the user&#39;s navigation of the distributed content in the user session. The EP gives the user the opportunity to rate distributed content. The EP saves the user&#39;s evaluation with the user session data and external stimuli data. The EP closes the user session when the user leaves the distributed content. The EP reopens the user session when the user returns to the distributed content.

CROSS-REFERENCE TO RELATED APPLICATION

The present invention is related to the subject matter of U.S. patentapplication Ser. No. 10/798,903, incorporated herein by reference.

FIELD OF THE INVENTION

The present invention is directed generally to a method for storingdistributed content evaluations and specifically to a method for storinga user's real-time evaluation of distributed content pages in a datarepository.

BACKGROUND OF THE INVENTION

Distributed content is a general term used to describe electronic mediathat is distributed to end users. Examples of distributed contentinclude webpages, websites, dynamically generated content, cellulartelephones using wireless application protocol (WAP) to serve content onthe cellular telephone screen, and so forth. Other examples ofdistributed content are known to persons of ordinary skill in the art.Because there is a high demand for adapting distributed content to theend users' needs, distributed content administrators (administrators)need high-quality feedback on the distributed content pages theyadminister. Due to the anonymity of distributed content users (users),reliable user feedback regarding the distributed content pages can bedifficult to obtain.

One of the problems associated with obtaining user evaluations ofdistributed content is that users do not give consistent feedback.Often, a user will only give feedback when the user has had aparticularly difficult time navigating the distributed content. Whilethese types of comments are useful to administrators in removingdistributed content that is difficult to use, such comments do notconvey information regarding other distributed content end users haveviewed. Therefore, a need exists for a method for a user to obtain userevaluations of all of the administered distributed content.

A second problem associated with user evaluation of distributed contentis that the user is sometimes presented with a single user evaluationform or survey to use in evaluating a plurality of distributed contentpages. When a user evaluates a plurality of distributed content pages ona single survey, the user tends to remember more information about themost recently navigated pages and less information about the firstpages. Thus, the survey does not adequately represent the user'sevaluation of the entire distributed content, but rather the user'sevaluation of the distributed content pages immediately preceding thesurvey. A survey that weighs the user's evaluation of more recentlynavigated content is called a back-loaded survey. Back-loaded surveysare not preferable because they do not adequately reflect the user'squalitative experience of the entire distributed content. Therefore, aneed exists for a method of capturing a user's evaluation of distributedcontent in which the user's evaluation evenly reflects the user'sexperience in navigation of the entire set of distributed content.

A third problem associated with user evaluation of distributed contentis that the survey is sometimes presented before the user has completedhis navigation of the distributed content. When the survey is placed atthe end of the user's navigation of the distributed content (i.e. afteruser selection of service, payment, and receipt of the confirmationnumber), users frequently do not complete the survey. Rather thancomplete the survey, the majority of users choose to close thedistributed content application. In order to increase the number ofcompleted surveys, administrators position the survey so that it appearsbefore the user has completed his navigation of the distributed content(i.e. after user selection of services but prior to payment). When asurvey is completed prior to conclusion of the user navigation of thedistributed content, the evaluation is said to be front-loaded.Front-loaded evaluations are not preferable because they do not capturea complete picture of the user's evaluation of the distributed content.Therefore, a need exists for a method of capturing a user's evaluationof distributed content after the user has completed his navigation ofthe distributed content.

In addition to the disadvantages discussed above, surveys also haveanother disadvantage: the survey is a standard document applied to awide variety of distributed content users. In other words, the surveyscannot be configured for specific users in the United States, Mexico,Asia, or Europe. The prior art surveys also cannot differentiate userswho view one version or type of the distributed content from users whoview another version or type of distributed content. If a survey wereable to differentiate between different types of users and thedistributed content they view or use, then the survey could becustomized for each type of user. Customizing the survey to each type ofuser would make the responses in the survey more meaningful. Therefore,a need exists for a method for surveying distributed content users inwhich the survey can be configured according to the characteristics andnavigation experiences of individual users or groups of users.

In addition to the disadvantages associated with surveys, it ispreferable to track the external stimuli presented to the user. Externalstimuli are data useful to distributed content administrators, but notdirectly related to the user's evaluation of the distributed content.One example of an external stimulus is the medium by which the useraccessed the distributed content. Possible medium sources include theWorld Wide Web, email, telephone, television, and so forth. Anotherexample of an external stimulus is the tactic used to contact or marketto the user. Persons of ordinary skill in the art have developed atechnique for tracking marketing tactics by assigning a unique ID toeach message provided to a user. The ability to store the externalstimuli data with the user survey would make the user surveys moremeaningful to distributed content administrators. Therefore, a needexists for a method for surveying distributed content in which theexternal stimuli presented to the user is captured in conjunction withthe user evaluation.

Consequently, a need exists in the art for an improved method for userevaluation of distributed content. A need exists for a method in whichthe user can identify the distributed content that is difficult to useand distributed content that is easy to use. A need exists for a methodof capturing a user's evaluation of distributed content in which theuser's evaluation evenly reflects the user's experience in navigation ofthe entire distributed content. A need exists for a method of capturinga user's evaluation of distributed content after the user has completedhis navigation of the distributed content. A need extends to a methodfor surveying distributed content users in which the survey can beconfigured for individual users. Finally, a need exists for a method forsurveying distributed content in which the external stimuli presented tothe user is captured in conjunction with the user evaluation.

SUMMARY OF THE INVENTION

The present invention, which meets the needs identified above, is a datarepository for storing a user evaluation of distributed content. Theuser evaluation is saved with other information such as the time anddate of the evaluation, the user's personal information, and thenavigation path the user used to access the distributed content page.The software embodiment of the present invention comprises an EvaluationProgram (EP) that creates a user session when a user accessesdistributed content. The EP records the user's navigation of thedistributed content in the user session. The EP gives the user theopportunity to rate distributed content if the distributed content pagehas a content rating window. The user also has the option to forgorating the distributed content page, if desired. If the user decides torate the distributed content page, the EP displays a content ratingwindow that allows the user to rate the distributed content page. The EPsaves the user's evaluation with the user session data and externalstimuli data. If the user accesses another distributed content page, theEP repeats the process described above. The EP closes the user sessionwhen the user leaves the distributed content. The EP reopens the usersession when the user returns to the distributed content.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the invention are setforth in the appended claims. The invention itself, however, as well asa preferred mode of use, further objectives and advantages thereof, willbest be understood by reference to the following detailed description ofan illustrative embodiment when read in conjunction with theaccompanying drawings, wherein:

FIG. 1 is an illustration of a computer network used to implement thepresent invention;

FIG. 2 is an illustration of a computer, including a memory and aprocessor, associated with the present invention;

FIG. 3 is an illustration of the logic of the Evaluation Program (EP) ofthe present invention;

FIG. 4 is an illustration of the content rating window of the presentinvention; and

FIG. 5 is an illustration of the data repository of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

As used herein, the term “computer” shall mean a machine having aprocessor, a memory, and an operating system, capable of interactionwith a user or other computer, and shall include without limitationdesktop computers, notebook computers, tablet computers, personaldigital assistants (PDAs), servers, handheld computers, and similardevices.

As used herein, the term “content rating window” shall mean a graphicaluser interface (GUI) that allows a user to rate a distributed contentpage.

As used herein, the term “distributed content” shall mean electroniccontent distributed to a plurality of end users over a computer network.Examples of distributed content include webpages and websites,dynamically generated content, and cellular telephones using wirelessapplication protocol (WAP) to serve content on the cellular telephonescreen. Other examples of distributed content are known to persons ofordinary skill in the art.

As used herein, the term “distributed content page” shall mean a singledistributed content document, file, script, view of content, ordatabase.

As used herein, the term “evaluate” shall mean for a user to rate thedistributed content page.

As used herein, the term “external stimuli” shall mean data regardingthe user's access of the distributed content that is not directlyrelated to the user's evaluation of the distributed content.

As used herein, the term “medium” shall mean the mechanism by which auser the distributed content. Possible medium sources include the WorldWide Web, email, telephone, and television.

As used herein, the term “navigation” shall mean to browse, selectoptions from, and/or click hyperlinks on a distributed content page.

As used herein, the term “tactic” shall mean a device or mechanism usedto contact or market to a user.

As used herein, the term “user ratings” shall mean a database containinga user evaluation of a distributed content page, the version of thedistributed content page, and external stimuli.

As used herein, the term “user session” shall mean a database of theuser information and the user's navigation history through a distributedcontent page.

FIG. 1 is an illustration of computer network 90 associated with thepresent invention. Computer network 90 comprises local computer 95electrically coupled to network 96. Local computer 95 is electricallycoupled to remote computer 94 and remote computer 93 via network 96.Local computer 95 is also electrically coupled to server 91 and database92 via network 96. Network 96 may be a simplified network connectionsuch as a local area network (LAN) or may be a larger network such as awide area network (WAN) or the Internet. Furthermore, computer network90 depicted in FIG. 1 is intended as a representation of a possibleoperating network containing the present invention and is not meant asan architectural limitation.

The internal configuration of a computer, including connection andorientation of the processor, memory, and input/output devices, is wellknown in the art. The present invention may be a method, a stand alonecomputer program, or a plug-in to an existing computer program. Personsof ordinary skill in the art are aware of how to configure computerprograms, such as those described herein, to plug into an existingcomputer program. Referring to FIG. 2, the methodology of the presentinvention is implemented on software by Evaluation Program (EP) 160. EP160 described herein can be stored within the memory of any computerdepicted in FIG. 1. Alternatively, EP 160 can be stored in an externalstorage device such as a removable disk, a CD-ROM, or a USB storagedevice. Memory 100 is illustrative of the memory within one of thecomputers of FIG. 1. Memory 100 also contains distributed content 120,content rating windows 140, and data repository 200.

Distributed content 120 is electronic content distributed to a pluralityof end users over a computer network. Distributed content 120 comprisesa plurality of distributed content pages. Examples of distributedcontent include webpages, websites, dynamically generated content, andcellular telephones using wireless application protocol (WAP) to servecontent on the cellular telephone screen. Other examples of distributedcontent are known to persons of ordinary skill in the art. Distributedcontent 120 contains at least one distributed content page accessible bya user. Content ratings windows 140 are windows that allow the users torate the distributed content pages. Data repository 200 stores the usersession data for the user, the user evaluations of the distributedcontent, and the external stimuli data. Data repository 200 includes theuser's personal information (i.e. if the user is male/female, the user'sphysical location, and so forth), the access time, the access date, theuser's IP address, the selected incentive plan, and whether the useraccesses the Internet via a computer, PDA, or cellular telephone. Datarepository 200 also includes user information such as the user'spurchase history and membership information. If desired, all or part ofthe user's information can be redacted prior to analysis by adistributed content administrator so that the user's privacy can bepreserved. The present invention may interface with distributed content120, content rating windows 140, and data repository 200 through memory100.

As part of the present invention, the memory 100 can be configured withEP 160, distributed content 120, content rating windows 140, and/or datarepository 200. Processor 106 can execute the instructions contained inEP 160. Processor 106 is also able to display data on display 102 andaccept user input on user input device 104. Processor 106, user inputdevice 104, display 102, and memory 100 are part of a computer such aslocal computer 95 in FIG. 1. Processor 106 can communicate with othercomputers via network 96.

In alternative embodiments, EP 160, distributed content 120, contentrating windows 140, and/or data repository 200 can be stored in thememory of other computers. Storing EP 160, distributed content 120,content rating windows 140, and/or data repository 200 in the memory ofother computers allows the processor workload to be distributed across aplurality of processors instead of a single processor. Furtherconfigurations of EP 160, distributed content 120, content ratingwindows 140, and/or data repository 200 across various memories, such asclient memory and server memory, are known by persons of ordinary skillin the art.

FIG. 3 is an illustration of the logic of Evaluation Program (EP) 160 ofthe present invention. EP 160 is a computer software program that allowsa user to evaluate a plurality of distributed content pages as the usercompletes his navigation of each distributed content page. EP 160 startswhenever the distributed content administrator invokes EP 160 (162). Auser then accesses a distributed content page (164). The distributedcontent page may be like one of the distributed content pages indistributed content 120 depicted in FIG. 2. EP 160 creates a usersession to track the user's navigation of the distributed content pages(166). The user session may be stored in data repository 200 depicted inFIG. 2. EP 160 then determines whether a content rating window has beencreated for the distributed content page (168). The content ratingwindow may be like content rating window 140 depicted in FIG. 2. If thedistributed content page does not have a content rating window, EP 160proceeds to step 180. If the distributed content page has a contentrating window, then EP 160 gives the user an opportunity to evaluate thecontent of the present distributed content page (170). EP 160 can givethe user the opportunity to rate the distributed content page bydisplaying a button that launches a content rating window.Alternatively, EP 160 can display the content rating window as a pop-upwindow or as a window adjacent to the distributed content page.Displaying the content rating window as a pop-up window or as anadjacent window allows the user to review the distributed content pagewhile completing the evaluation form on the content rating window.

EP 160 then makes a determination whether the user wants to rate thedistributed content page (172). The user can indicate that he wants torate the distributed content page by clicking the button to launch thecontent rating window or by rating the distributed content page on thecontent rating window. The user can indicate that he does not want torate the distributed content page by not clicking the button to launchthe content rating window or by closing the content rating windowwithout evaluating the content. If the user does not want to rate thedistributed content page, EP 160 proceeds to step 180. If the user wantsto rate the distributed content page, then EP 160 displays the contentrating window, if not already displayed (174). The present inventiondoes not need to display the content rating window if the content ratingwindow was displayed as part of step 170. The user then rates thepresent distributed content page (176). In evaluating the distributedcontent page, the user completes a user rating file by answering aplurality of questions regarding the distributed content page. The userhas the option of entering a message in the comments area of the contentrating window. If desired, the user can save the user rating file inmemory and access the user rating file at a later date. The user cancomplete his user rating file via email, web browser, telephone, or anyother communicative means. Persons of ordinary skill in the art areaware of how to access a computer file, such as a user rating file, viaemail, web browser, telephone, and other communicative means. EP 160then saves the user rating file with a copy of the distributed contentpage and the user session data (178). EP 160 then proceeds to step 180.

At step 180, EP 160 then determines whether the user has accessed a newdistributed content page (180). If the user has accessed a newdistributed content page, then EP 160 returns to step 168. If the userhas not accessed a new distributed content page, then EP 160 closes theuser session and saves the user session in the user sessions file (182).EP 160 then ends (184). When the user returns to the distributedcontent, EP 160 reopens the user session and continues to track theuser's access throughout the distributed content. Alternately, a usermay close a session and resume a second session. Because sessions aretracked to unique users, an end user is capable of providing ratings ofdistributed content on a per-session basis, or the user may rate some orall content viewed across multiple sessions. Maintaining user sessionsfor a single user allows the present invention to develop a moreaccurate history of a specific user's navigation through the distributedcontent.

FIG. 4 is an illustration of one embodiment of content rating window140. Content rating window 140 allows the user to rate distributedcontent while the user is navigating the distributed content page.Content rating window 140 asks the user a series of questions 142. Theuser enters the answers 144 to the questions 142. The user may alsoenter comments 146, if desired. The user may submit the evaluation usingthe “Submit” button.

FIG. 5 is an illustration of one embodiment of data repository 200. Datarepository 200 comprises user ID 202, user IP address 204, distributedcontent page 206, distributed content page version 208, accessed via210, time/date 212, duration 214, exited via 216, user rating 218,content rating window version 220, session ID 222, medium 224, andtactic ID 226. User ID 202 identifies the specific user and mayoptionally reference the user's personal information if such informationis stored in a database associated with the present invention. Theuser's personal information may be redacted, if desired. User IP address204 identifies the IP address for the user. Distributed content page 206is the distributed content page that the user accessed. Distributedcontent page version 208 is the version of distributed content page 206.Accessed via 210 is the path by which the user accessed distributedcontent page 206. Time/date 212 is the time and date that the useraccessed distributed content page 206. Duration 214 is the total timethe user spent browsing distributed content page 206. Exited via 216 isthe path by which the user exited distributed content page 206. Userrating 218 is a Boolean field that defines whether the user completed auser rating for distributed content page 206. Content rating windowversion 220 is the version of the content rating window 140 that theuser viewed to evaluate distributed content 206. Session ID 222 is theidentification number for the user's navigational session of distributedcontent. When the user opens an application or logs on after a previoussession, the present invention creates a new session ID. Medium 224 isthe distributed content medium by which the user accessed thedistributed content. Tactic ID 226 is the unique number that identifiesthe message(s) sent to the user. Medium 224 and tactic ID 226 areexamples of external stimuli. Persons of ordinary skill in the art areaware of other types of external stimuli that can be added to datarepository 200.

The configuration of EP 160, distributed content 120, content ratingwindows 140, and data repository 200 of the present invention offer manyadvantages over the prior art solutions. For example, because the userratings are saved in conjunction with specific information about theuser in data repository 200 may be categorized by any of the fields indata repository 200. The present invention also resolves the problem offront-loaded and back-loaded evaluations by gathering information withinthe context of a complete visit to the distributed content page by theuser. The present invention provides the user with an opportunity toevaluate a plurality of distributed content pages within a plurality ofdifferent types of distributed content. Through the incentive program,the present invention encourages user evaluation of the distributedcontent pages. The users are able to refresh their memory about thedistributed content page by flipping back and forth between thedistributed content page and content rating window 140 while evaluatingthe distributed content page.

The present invention is also extensible. The invention allows theadministrators to analyze the duration data in data repository 200 todifferentiate between distributed content page requests created by straymouse clicks (or other pointing device) and deliberate distributedcontent page requests. The present invention allows the user to launchand re-launch content rating window 140 when desired. The presentinvention can be configured to allow a user to update his evaluation byreopening his user rating. The user can then complete his users ratingvia email, web browser, telephone, or any other communicative means. Thepresent invention allows for integration of a company's complaintmanagement, support, and similar systems. Finally, the present inventioncan be cross-referenced with other survey data.

With respect to the above description, it is to be realized that theoptimum dimensional relationships for the parts of the invention, toinclude variations in size, materials, shape, form, function, manner ofoperation, assembly, and use are deemed readily apparent and obvious toone of ordinary skill in the art. The present invention encompasses allequivalent relationships to those illustrated in the drawings anddescribed in the specification. The novel spirit of the presentinvention is still embodied by reordering or deleting some of the stepscontained in this disclosure. The spirit of the invention is not meantto be limited in any way except by proper construction of the followingclaims.

What is claimed is:
 1. A program product for storing in a datarepository a plurality of evaluations from a plurality of users whoevaluate a distributed content page using a content rating window, theprogram product comprising: a computer-usable medium; wherein thecomputer usable medium comprises instructions for a computer to performsteps comprising: accepting access by one of the plurality of users tothe distributed content page; creating a user session; accepting a userevaluation of the distributed content page; and saving the userevaluation as a user rating in the data repository, wherein the userrating is cross-referenced with the user session and an externalstimuli; wherein the content rating window is a graphical user interfacethat allows the user to rate the distributed content page; wherein theevaluations are stored with a version of the distributed content, aversion of the content rating window, and the external stimuli presentedto the users; and wherein the distributed content page is displayed on aportable electronic device.
 2. The program product of claim 1 whereinthe user session tracks the user's navigation of a plurality of thedistributed content pages.
 3. The program product of claim 2 wherein thecontent rating window is based on the characteristics of each user. 4.The program product of claim 3 wherein the evaluation program furtherperforms steps comprising: determining whether the distributed contentpage is associated with the content rating window; responsive to thedetermination that the distributed content page is associated with thecontent rating window, giving the user the opportunity to evaluate thedistributed content page; determining whether the user desires toevaluate the distributed content page; and responsive to thedetermination that the user desires to evaluate the distributed contentpage, displaying the content rating window.
 5. The program product ofclaim 4 wherein the evaluation program further performs stepscomprising: responsive to the determination that the distributed contentpage is not associated with the content rating window, determiningwhether the user has accessed a different distributed content page. 6.The program product of claim 5 wherein the evaluation program furtherperforms steps comprising: responsive to the determination that the userdoes not desire to evaluate the distributed content page, determiningwhether the user has accessed the different distributed content page. 7.The program product of claim 6 wherein the evaluation program furtherperforms steps comprising: responsive to the determination that the userhas accessed the different distributed content page, repeating the stepsin claim
 6. 8. The program product of claim 7 wherein the evaluationprogram further performs steps comprising: responsive to thedetermination that the user has not accessed the different distributedcontent page, closing the user session.
 9. The program product of claim8 wherein the user's personal information can be redacted from the datarepository.
 10. The program product of claim 1 wherein the user savesthe user rating in a memory and completes the user rating at a laterdate.
 11. The program product of claim 1 wherein the user rating may becategorized by any of the fields in the user session, the user rating,the version of the distributed content, the version of the contentrating window, or the external stimuli.
 12. The program product of claim1 wherein the user completes the user rating by email, web browser, ortelephone.
 13. The program product of claim 1 wherein the user ratinggathers evaluative information from the user based on the user'scomplete navigation of the plurality of the distributed content pages.14. The program product of claim 1 wherein the user rating allows theuser to evaluate the plurality of the distributed content pages.
 15. Theprogram product of claim 1 wherein the user reviews the distributedcontent page simultaneous with reviewing the content rating window. 16.The program product of claim 1 wherein distributed content administratorcan distinguish between an accidental distributed content page requestand an intentional distributed content page request by analyzing aduration data in the user session.
 17. The program product of claim 1wherein the distributed content page is a webpage.
 18. An apparatus forstoring in a data repository a plurality of evaluations from a pluralityof users who evaluate a distributed content page using a content ratingwindow, the apparatus comprising: means for accepting access by one ofthe plurality of users to the distributed content page; means forcreating a user session; means for accepting a user evaluation of thedistributed content page; means for saving the user evaluation as a userrating in the data repository, wherein the user rating iscross-referenced with the user session and the external stimuli; whereinthe evaluations are stored with a version of the distributed content, aversion of the content rating window, and an external stimuli presentedto the users; wherein the user session tracks the user's navigation of aplurality of the distributed content pages; wherein the content ratingwindow is based on the characteristics of each user; determining whetherthe distributed content page is associated with the content ratingwindow; responsive to the determination that the distributed contentpage is associated with the content rating window, giving the user theopportunity to evaluate the distributed content page; determiningwhether the user desires to evaluate the distributed content page;responsive to the determination that the user desires to evaluate thedistributed content page, displaying the content rating window;responsive to the determination that the distributed content page is notassociated with the content rating window, determining whether the userhas accessed a different distributed content page; responsive to thedetermination that the user does not desire to evaluate the distributedcontent page, determining whether the user has accessed the differentdistributed content page; responsive to the determination that the userhas accessed the different distributed content page, determining whetherthe different distributed content page has a content rating window;responsive to the determination that the user has not accessed thedifferent distributed content page, closing the user session; whereinthe user's personal information can be redacted from the datarepository; wherein the user saves the user rating in a memory andcompletes the user rating at a later date; wherein the user rating maybe categorized by any of the fields in the user session, the userrating, the version of the distributed content, the version of thecontent rating window, or the external stimuli; wherein the usercompletes the user rating by email, web browser, or telephone; whereinthe user rating gathers evaluative information from the user based onthe user's complete navigation of the plurality of the distributedcontent pages; wherein the user rating allows the user to evaluate theplurality of the distributed content pages; wherein the user reviews thedistributed content page simultaneous with reviewing the content ratingwindow; and wherein distributed content administrator can distinguishbetween an accidental distributed content page request and anintentional distributed content page request by analyzing a durationdata in the user session.
 19. The apparatus of claim 18 wherein thedistributed content page is a webpage.
 20. The apparatus of claim 18wherein the distributed content page is displayed on a portableelectronic device.